Image processing apparatus, image capturing apparatus, image processing method, control method, and storage medium

ABSTRACT

There is provided an image processing apparatus. An obtaining unit obtains an image, and first NR information defining noise reduction processing using a first noise reduction processing method which an external apparatus can apply to the image. A generating unit generates, on the basis of the first NR information, second NR information defining noise reduction processing using a second noise reduction processing method which is different from the first noise reduction processing method. A noise reducing unit applies the noise reduction processing using the second noise reduction processing method to the image in accordance with the second NR information.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing apparatus, an imagecapturing apparatus, an image processing method, a control method, and astorage medium.

Description of the Related Art

Image capturing apparatuses such as video cameras can carry out adeveloping process on raw image data (RAW images) captured by an imagesensor during shooting. Specifically, the image capturing apparatuscarries out a debayering process (demosaicing) on the RAW image andconverts the RAW image into signals corresponding to luminance and colordifference components. The image capturing apparatus also carries outprocesses such as noise reduction, optical distortion correction, andimage optimization on each signal. The image capturing apparatus thencompresses/encodes the luminance signal and color difference signals,and records these in a recording medium.

With respect to noise reduction processing, Japanese Patent Laid-OpenNo. 2010-258996 proposes obtaining a frame image, analyzing abright/dark histogram distribution of the frame image, and carrying outnoise reduction processing on the basis of bright/dark distributioninformation.

There are also image capturing apparatuses which can record moving imagedata in the RAW format (“RAW moving images” hereinafter). Although a RAWmoving image involves recording a much greater amount of data, a RAWmoving image is advantageous in that it keeps distortion, degradation,and the like to a minimum compared to the original image and canfurthermore be edited after shooting, and is therefore widely used inthe cinema industry and the like. With image capturing apparatuses thatrecord RAW moving images, a technique is known in which developingparameters are recorded in association with each frame along with theRAW moving image, and during playback, the RAW moving image isdeveloped/played back using the developing parameters. Developingparameters used when generating a monitoring video for the purpose ofchecking the angle of view and the like, developing parameters for proxyrecording videos recorded at the same time as the RAW moving image, andso on are used as the developing parameters which are recorded whenshooting a RAW moving image. Parameters for various types of imageprocessing, such as noise reduction processing, edge enhancement, andwhite balance, may also be recorded. The recorded RAW moving image isloaded into a PC, for example, and subjected to developing processesusing various types of developing applications.

There has been a problem in that if the noise reduction processingmethod used by the developing application is different from the noisereduction processing method used by the image capturing apparatus, theparameters for the noise reduction processing which have been recorded(noise reduction processing information) cannot be used when playingback the image. For example, there are cases where the noise reductionprocessing method used by the image capturing apparatus is a framereference-based, so-called “time direction” noise reduction processingmethod, whereas the noise reduction processing method used by thedeveloping application is a so-called “spatial filter-based” noisereduction processing method. In such a case, the parameters for noisereduction processing which have been recorded cannot be used whenplaying back images.

SUMMARY OF THE INVENTION

Having been achieved in light of such circumstances, the presentinvention provides a technique that makes it possible to use noisereduction processing information, which defines noise reductionprocessing using a specific noise reduction processing method, whencarrying out noise reduction processing using a different noisereduction processing method.

According to a first aspect of the present invention, there is providedan image processing apparatus comprising at least one processor and/orat least one circuit which functions as: an obtaining unit configured toobtain an image, and first NR information defining noise reductionprocessing using a first noise reduction processing method which anexternal apparatus can apply to the image; a generating unit configuredto generate, on the basis of the first NR information, second NRinformation defining noise reduction processing using a second noisereduction processing method which is different from the first noisereduction processing method; and a noise reducing unit configured toapply the noise reduction processing using the second noise reductionprocessing method to the image in accordance with the second NRinformation.

According to a second aspect of the present invention, there is providedan image capturing apparatus comprising: an image sensor configured tocapture an image; and at least one processor and/or at least one circuitwhich functions as: a first generating unit configured to generate firstNR information defining noise reduction processing using a first noisereduction processing method; a second generating unit configured togenerate, on the basis of the first NR information, second NRinformation defining noise reduction processing using a second noisereduction processing method which is different from the first noisereduction processing method and which an external apparatus can apply tothe image; a noise reducing unit configured to apply the noise reductionprocessing using the first noise reduction processing method to theimage in accordance with the first NR information; and a recording unitconfigured to record the second NR information, and the image to whichthe noise reduction processing has not yet been applied by the noisereducing unit, in association with each other.

According to a third aspect of the present invention, there is providedan image processing method executed by an image processing apparatus,comprising: obtaining an image, and first NR information defining noisereduction processing using a first noise reduction processing methodwhich an external apparatus can apply to the image; generating, on thebasis of the first NR information, second NR information defining noisereduction processing using a second noise reduction processing methodwhich is different from the first noise reduction processing method; andapplying the noise reduction processing using the second noise reductionprocessing method to the image in accordance with the second NRinformation.

According to a fourth aspect of the present invention, there is provideda control method executed by an image capturing apparatus comprising:capturing an image; generating first NR information defining noisereduction processing using a first noise reduction processing method;generating, on the basis of the first NR information, second NRinformation defining noise reduction processing using a second noisereduction processing method which is different from the first noisereduction processing method and which an external apparatus can apply tothe image; applying the noise reduction processing using the first noisereduction processing method to the image in accordance with the first NRinformation; and recording the second NR information, and the image towhich the noise reduction processing has not yet been applied by thenoise reducing unit, in association with each other.

According to a fifth aspect of the present invention, there is provideda non-transitory computer-readable storage medium which stores a programfor causing a computer to execute an image processing method comprising:obtaining an image, and first NR information defining noise reductionprocessing using a first noise reduction processing method which anexternal apparatus can apply to the image; generating, on the basis ofthe first NR information, second NR information defining noise reductionprocessing using a second noise reduction processing method which isdifferent from the first noise reduction processing method; and applyingthe noise reduction processing using the second noise reductionprocessing method to the image in accordance with the second NRinformation.

According to a sixth aspect of the present invention, there is provideda non-transitory computer-readable storage medium which stores a programfor causing a computer to execute a control method comprising: capturingan image; generating first NR information defining noise reductionprocessing using a first noise reduction processing method; generating,on the basis of the first NR information, second NR information definingnoise reduction processing using a second noise reduction processingmethod which is different from the first noise reduction processingmethod and which an external apparatus can apply to the image; applyingthe noise reduction processing using the first noise reductionprocessing method to the image in accordance with the first NRinformation; and recording the second NR information, and the image towhich the noise reduction processing has not yet been applied by thenoise reducing unit, in association with each other.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the internal configuration of adigital video camera 100.

FIG. 2 is a block diagram illustrating the internal configuration of animage playback apparatus 200.

FIG. 3 is a diagram illustrating, in detail, the configuration of animage processing circuit 24 and a system control circuit 50, accordingto a first embodiment.

FIG. 4 is a conceptual diagram illustrating a RAW moving image to whichmetadata has been added, according to the first embodiment.

FIG. 5 is a flowchart illustrating an NR information setting processexecuted by the image playback apparatus 200.

FIG. 6 is a diagram illustrating a relationship between a RAW movingimage and metadata (first NR information), and second NR information anda playback moving image.

FIG. 7 is a diagram illustrating, in detail, the configuration of theimage processing circuit 24 and the system control circuit 50, accordingto a second embodiment.

FIG. 8 is a flowchart illustrating an NR information recording processexecuted by the digital video camera 100.

FIG. 9 is a conceptual diagram illustrating a RAW moving image to whichmetadata has been added, according to the second embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

First Embodiment

FIG. 1 is a block diagram illustrating the internal configuration of adigital video camera 100 serving as an example of an image capturingapparatus. In FIG. 1, an image capturing lens 103 is a lens groupincluding a zoom lens and a focus lens, which forms a subject image. Anaperture stop 101 is an aperture stop used to adjust the light amount.An ND filter 104 is a filter used for light reduction. An image sensor22 is constituted by a CCD, a CMOS element, or the like that converts anoptical image into an electrical signal. Additionally, the image sensor22 has functions for controlling charge accumulation using an electronicshutter, changing the analog gain and readout speed, and so on. An A/Dconverter 23 converts analog signals into digital signals. The A/Dconverter 23 is used to convert analog signals output from the imagesensor 22 into digital signals. A barrier 102 prevents an imagecapturing system of the digital video camera 100 including the imagecapturing lens 103, the aperture stop 101, and the image sensor 22 frombeing soiled or damaged by covering the image capturing system includingthe image capturing lens 103.

The present embodiment will describe an example in which the digitalvideo camera 100 records a RAW moving image. An image processing circuit24 carries out processing such as color conversion processing, gammacorrection, noise reduction processing, adding digital gain, and thelike on data from the A/D converter 23 or data from a memory controlcircuit 15, in order to generate a monitoring video. However, the imageprocessing circuit 24 does not process a RAW moving image which is arecorded video. The image processing circuit 24 also carries outpredetermined computational processes using captured image data, andsends the computation results to a system control circuit 50. On thebasis of the computation results which have been sent, the systemcontrol circuit 50 carries out exposure control, rangefinding control,white balance control, and the like. Through-the-lens (TTL) autofocus(AF) processing, automatic exposure (AE) processing, auto white balance(AWB) processing, and so on are carried out as a result. The imageprocessing circuit 24 will be described in detail later.

Data output from the A/D converter 23 is written directly into memory 32through the image processing circuit 24 and the memory control circuit15, or through the memory control circuit 15. The memory 32 stores theimage data captured by the image sensor 22 and converted into digitaldata by the A/D converter 23, image data for display in a display 28,and the like. The memory 32 is provided with a storage capacitysufficient to store a predetermined time's worth of moving images andaudio. The memory 32 also functions as image display memory (videomemory).

A D/A converter 13 converts image display data, stored in the memory 32,into an analog signal and supplies the analog signal to the display 28.The image data for display written into the memory 32 is displayed bythe display 28 via the D/A converter 13 in this manner. The display 28carries out a display in a display device such as an LCD or the likebased on the analog signal from the D/A converter 13. The digitalsignals A/D-converted by the A/D converter 23 and accumulated in thememory 32 are converted to analog signals by the D/A converter 13 andare then sequentially transferred to and displayed by the display 28; asa result, the display 28 can function as an electronic viewfinder, and athrough-the-lens image can be displayed.

Non-volatile memory 56 is electrically erasable/recordable memory, e.g.,EEPROM. Operational constants, programs, and so on of the system controlcircuit 50 are stored in the non-volatile memory 56. “Programs” refersto programs through which the system control circuit 50 executes varioustypes of processes.

The system control circuit 50 controls the digital video camera 100 as awhole. The system control circuit 50 implements the various types ofprocesses by executing programs recorded in the non-volatile memory 56mentioned above. System memory 52 is RAM, for example. Operationalconstants and variables of the system control circuit 50, programs readout from the non-volatile memory 56, and so on are loaded into thesystem memory 52. The system control circuit 50 also carries out displaycontrol by controlling the memory 32, the D/A converter 13, the display28, and so on.

A system timer 53 measures times used in various types of control,measures the time of an internal clock, and so on. A mode changingswitch 60, a record switch 61, and an operation member 70 are operationmeans for inputting various types of operational instructions to thesystem control circuit 50.

The mode changing switch 60 switches an operating mode of the systemcontrol circuit 50 among a moving image recording mode, a still imagerecording mode, a playback mode, and so on. The moving image recordingmode and still image recording mode include an auto shooting mode, anauto scene judgment mode, a manual mode, various types of scene modeshaving shooting settings for each type of shot scene, a program AE mode,a custom mode, and so on. The mode changing switch 60 can switch theoperating mode directly to one of these modes included in the movingimage recording mode. Alternatively, the mode switch may be carried outby first switching to the moving image recording mode using the modechanging switch 60 and then using other operating means to switch to oneof the modes included in the moving image recording mode. The recordswitch 61 switches between a shooting standby state and a shootingstate. The system control circuit 50 starts a sequence of operations,from reading out signals from the image sensor 22 to writing movingimage data into a recording medium 90, in response to the record switch61 being operated.

Functions relevant for different situations are assigned to theoperation member 70, which then act as various types of functionbuttons, by making an operation for selecting various types of functionicons displayed in the display 28. An end button, a return button, anext image button, a jump button, a sort button, an attribute changebutton, and so on are examples of the function buttons. For example, amenu screen in which various types of settings can be made is displayedin the display 28 when a menu button is pressed. A user can make varioustypes of settings intuitively using the menu screen displayed in thedisplay 28, along with four directional (up, down, left, and right)cross key, a set button, and so on.

A power control circuit 80 is constituted by a battery detectioncircuit, a DC-DC converter, switch circuits for switching the blocksthrough which power passes, and so on, and detects whether or not abattery is connected, the type of the battery, the remaining batterypower, and so on. The power control circuit 80 also controls the DC-DCconverter on the basis of the detection results and instructions fromthe system control circuit 50, and supplies a necessary voltage for anecessary period to the various units, including the recording medium90. A power source 30 is a primary battery such as an alkaline battery,a lithium battery, or the like, a secondary battery such as a NiCdbattery, a NiMH battery, a Li ion battery, or the like, an AC adapter,and so on. An I/F 18 is an interface for the recording medium 90, whichis a memory card, a hard disk, or the like, or for an external outputdevice. FIG. 1 illustrates a state in which the recording medium 90 isconnected. The recording medium 90 is a recording medium for recordingshot RAW moving images, such as a memory card or the like, and isconstituted by semiconductor memory, a magnetic disk, or the like.

Next, the processing performed by the image processing circuit 24 andthe system control circuit 50 will be described in detail with referenceto FIG. 3. Note that the image processing circuit 24 is capable ofobtaining various types of information from within the digital videocamera 100, including exposure parameters such as the f-stop value, NDinformation, and shutter speed, through the system control circuit 50.

A first NR circuit 301 carries out recursive noise reduction processing,which is noise reduction processing in the time direction, on themonitoring video.

A first NR information generating circuit 302 generates parameters(first NR information), such as a recursive coefficient, which definethe noise reduction processing executed by the first NR circuit 301, andsets the generated parameters in the first NR circuit 301. The strengthof the recursive noise reduction processing is generally known to beexpressed by the recursive coefficient. The recursive coefficient is anumerical value indicating the degree to which the effect of differencesbetween frames is to be eliminated, with higher numerical valuesindicating higher noise reduction effects (i.e., higher noise reductionstrengths). For example, the first NR information generating circuit 302holds a plurality of parameters corresponding to strengths of the noisereduction processing in advance, and then generates the first NRinformation in accordance with differences in the signal level betweenframes or a strength setting made through the operation member 70.

A metadata generating circuit 304 generates metadata to be added to theRAW moving image. The metadata includes the first NR information andvarious types of developing parameters. When generating the metadata,the metadata generating circuit 304 adds information indicating the typeof the noise reduction processing executed by the first NR circuit 301to the first NR information. An adder 305 adds the metadata generated bythe metadata generating circuit 304 to the RAW moving image input to theimage processing circuit 24.

Note that the type of the noise reduction processing executed by thefirst NR circuit 301 (the noise reduction processing method) is notlimited to recursive noise reduction processing (a recursive-type noisereduction processing method). The specific content of the first NRinformation changes depending on the type of the noise reductionprocessing executed by the first NR circuit 301, but includesinformation that can specify the content of the noise reductionprocessing, e.g., the type, processing order, various setting values,and so on.

FIG. 4 is a conceptual diagram of a RAW moving image to which themetadata has been added. A RAW moving image 401 is constituted by threeframes' worth of images (frames 1 to 3). Metadata 402 includesparameters for the noise reduction processing to be applied, on aframe-by-frame basis. In the following descriptions, R1 represents arecursive coefficient 2, and R2 represents a recursive coefficient 4. Asdescribed earlier, the strength of the noise reduction processing is seton the basis of the difference in signal levels between frames orthrough the operation member 70, and parameters based on the setstrength are recorded for each frame.

Next, an image playback apparatus 200, serving as an example of an imageprocessing apparatus, will be described with reference to FIG. 2. FIG. 2is a block diagram illustrating the internal configuration of the imageplayback apparatus 200. Video data 201 includes the RAW moving image tobe played back. The image playback apparatus 200 can obtain the videodata 201, which includes a RAW moving image shot by the digital videocamera 100, from the aforementioned recording medium 90 (FIG. 1), forexample. Metadata 202 is the metadata generated by the aforementionedmetadata generating circuit 304 (FIG. 2).

An image processing circuit 203 includes a developing processing circuit204 and a second NR circuit 205, and carries out various types of imagequality processing on the RAW moving image, including noise reductionprocessing. Although white balance control, sharpness control, and thelike are also typically carried out by the image processing circuit 203,such control is not necessary for the descriptions of the presentembodiment, and thus configurations for white balance control, sharpnesscontrol, and the like are not illustrated in the drawings. Thedeveloping processing circuit 204 carries out a developing process onthe RAW moving image. For example, the developing processing circuit 204carries out a debayering process on the RAW moving image, converting theRAW moving image into a predetermined video signal format such as theso-called YUV format, constituted by luminance and color differencesignals. The second NR circuit 205 carries out epsilon filterprocessing, which is spatial filter-based noise reduction processing, onthe developed video. The strength of the epsilon filter typicallyincreases the weighting ratio of the surrounding pixel regions andsurrounding pixels on a pixel of interest. However, to simplify thedescriptions, the present embodiment will describe a configuration inwhich the noise reduction effect is strengthened by broadening areference pixel range to 3×3 pixels, 5×5 pixels, and so on, as anexample.

Note that the type of the noise reduction processing executed by thesecond NR circuit 205 (the noise reduction processing method) is notlimited to an epsilon filter process. There are a variety of types ofnoise reduction processing, such as median filter processing, Gaussianfilter processing, and so on, but any types may be used as long as thefirst NR circuit 301 and the second NR circuit 205 carry out differenttypes of noise reduction processing, and thus the present embodiment isnot limited to a specific type. For example, the second NR circuit 205uses one of a recursive noise reduction processing method and a spatialfilter-type noise reduction processing method, and the first NR circuit301 uses the other of the recursive noise reduction processing methodand the spatial filter-type noise reduction processing method.

Additionally, although the present embodiment describes a case where theimage to be played back is a RAW moving image as an example, the imageto be played back is not limited to a RAW moving image. For example, theimage to be played back may be a moving image that has already beendeveloped. In this case, the developing processing circuit 204 does notcarry out the developing process. Furthermore, the image to be playedback may be a RAW still image, or a still image that has already beendeveloped. The technique of the present embodiment can be applied to anysituation in which it is necessary to change the type, strengthsettings, and so on of noise reduction processing between when an imageis shot and when the image is played back.

A metadata obtaining circuit 206 obtains the metadata 202 associatedwith each frame of the video data 201. A first NR informationdetermining circuit 207 determines information pertaining to the noisereduction processing that was carried out by the digital video camera100 when shooting the RAW moving image (the first NR information) on thebasis of the metadata 202. Note that the RAW moving image to be playedback has not yet been subjected to noise reduction processing. The“noise reduction processing carried out by the digital video camera 100when shooting the RAW moving image” refers to the noise reductionprocessing carried out to generate the monitoring video when shootingthe RAW moving image.

A second NR information generating circuit 208 generates noise reductionprocessing information for spatial filter-type noise reductionprocessing which can be used by the second NR circuit 205 (the second NRinformation) on the basis of the first NR information.

A control circuit 209 includes ROM and RAM (not shown), and controls thevarious units in the image playback apparatus 200 by using the RAM aswork memory to execute programs stored in the ROM.

An NR information setting process executed by the image playbackapparatus 200 will be described next with reference to FIG. 5. The imageplayback apparatus 200 executes the processing illustrated in thisflowchart on a frame-by-frame basis upon the video data 201 and themetadata 202 being input to the image playback apparatus 200.

In step S501, the metadata obtaining circuit 206 obtains the metadata202 and outputs that metadata 202 to the first NR informationdetermining circuit 207.

In step S502, the first NR information determining circuit 207determines whether or not the first NR information can be obtained fromthe metadata 202. If the metadata 202 contains the first NR information,the first NR information determining circuit 207 determines that thefirst NR information can be obtained from the metadata 202. Note thateven if the metadata 202 does not contain the first NR information,there are situations where the first NR information can be derived fromsome kind of information contained in the metadata 202. The first NRinformation determining circuit 207 determines that the first NRinformation can be obtained from the metadata 202 in such situations aswell. The information which enables the first NR information to bederived is, for example, camera settings such as ISO sensitivity,subject luminance information, and the like, for example. For example,the first NR information determining circuit 207 holds table data inwhich details such as camera settings, luminance information, and thelike are associated with details of the noise reduction processing, andderives the first NR information from the camera settings, luminanceinformation, and the like by referring to the table data. If it isdetermined that the first NR information can be obtained from themetadata 202, the process moves to step S504, and if not, the processmoves to step S503.

In step S503, the first NR information determining circuit 207 sets thenoise reduction processing of the second NR circuit 205 to “off” via thesecond NR information generating circuit 208.

In step S504, the first NR information determining circuit 207 outputsthe first NR information to the second NR information generating circuit208, and the second NR information generating circuit 208 determineswhether or not it is necessary to convert the first NR information. Thesecond NR information generating circuit 208 determines that it isnecessary to convert the first NR information if the type of the noisereduction processing corresponding to the first NR information isdifferent from the type of the noise reduction processing executed bythe second NR circuit 205. On the other hand, the second NR informationgenerating circuit 208 determines that it is not necessary to convertthe first NR information if the type of the noise reduction processingcorresponding to the first NR information is the same as the type of thenoise reduction processing executed by the second NR circuit 205. If itis not necessary to convert the first NR information, the process movesto step S505, whereas if it is necessary to convert the first NRinformation, the process moves to step S506.

In step S505, the first NR information determining circuit 207 sets thefirst NR information in the second NR circuit 205. As a result, thesecond NR circuit 205 executes the same noise reduction processing asthat which the digital video camera 100 carried out on the monitoringvideo when shooting the RAW moving image.

In step S506, the second NR information generating circuit 208 generatesthe noise reduction processing information for the noise reductionprocessing executed by the second NR circuit 205 (the second NRinformation) on the basis of the first NR information. As mentionedearlier, the present embodiment describes a case where the first NRcircuit 301 executes recursive noise reduction processing and the secondNR circuit 205 executes epsilon filter processing as an example. Thesecond NR information generating circuit 208 generates the second NRinformation by converting setting values for the recursive noisereduction processing into setting values for the epsilon filterprocessing. For converting the setting values, the second NR informationgenerating circuit 208 holds, in advance, table data definingassociations between setting values which produce similar noisereduction effects and which have a permissible level of negativeeffects, for example. Generally speaking, afterimages in the movingimages are a negative effect of recursive noise reduction processing,whereas a drop in resolution is a negative effect of spatial filter-typenoise reduction processing. Accordingly, the table data to be preparedis determined by confirming the video in advance, for example. Forexample, it is assumed that the second NR information generating circuit208 holds table data such as R1 (weak; recursive coefficient 2)→P1(weak: 3×3 pixels), and R2 (strong: recursive coefficient 4)→P2 (strong:5×5 pixels).

In step S507, the second NR information generating circuit 208 sets thegenerated second NR information in the second NR circuit 205. Throughthis, a noise reduction effect similar to the effect of the noisereduction processing which has been carried out by the digital videocamera 100 on the monitoring video when shooting the RAW moving image isalso achieved in the moving image played back by the image playbackapparatus 200.

Here, a configuration has been described in which the second NRinformation is generated so that the effect of the noise reductionprocessing executed by the second NR circuit 205 according to the secondNR information becomes similar to the effect of the noise reductionprocessing executed by the first NR circuit 301 according to the firstNR information. However, in the present embodiment, it is not absolutelynecessary for the effect of the noise reduction processing executed bythe second NR circuit 205 according to the second NR information tobecome similar to the effect of the noise reduction processing executedby the first NR circuit 301 according to the first NR information.Regardless of the format, an effect in which the first NR informationcan be utilized in the execution of the noise reduction processing bythe second NR circuit 205 is achieved as long as the second NRinformation is generated on the basis of the first NR information.

Additionally, the descriptions of the present embodiment assume that thefirst NR circuit 301 actually carries out the noise reduction processingin accordance with the first NR information. However, it is notabsolutely necessary for the first NR circuit 301 to carry out the noisereduction processing in accordance with the first NR information. Theeffects of the present embodiment can be achieved as long as the imageplayback apparatus 200 is configured so that the second NR informationis generated on the basis of the first NR information, which definesnoise reduction processing carried out through a specific noisereduction processing method applicable to an external apparatus such asthe digital video camera 100.

FIG. 6 is a diagram illustrating a relationship between a RAW movingimage and metadata (the first NR information), and the second NRinformation and a playback moving image. Second NR information 603 isthe second NR information generated in step S506, and as describedearlier, P1 is a setting with a weak noise reduction effect for areference region of 3×3 pixels, whereas P2 is a setting with a strongnoise reduction effect for a reference region broader than that of P1,i.e., 5×5 pixels. As described earlier, the processing in the flowchartillustrated in FIG. 5 is executed for each frame, and thus the second NRinformation contains strength setting parameters for each of the frames.Playback moving image 604 contains playback frames that have beensubjected to the noise reduction processing in accordance with thesecond NR information.

According to the first embodiment as described thus far, the imageplayback apparatus 200 obtains the first NR information, which definesnoise reduction processing applied to a RAW moving image for the digitalvideo camera 100 to generate the monitoring video. Then, on the basis ofthe first NR information, the image playback apparatus 200 generates thesecond NR information, which defines noise reduction processing using anoise reduction processing method different from that used by thedigital video camera 100, and applies the noise reduction processing tothe RAW moving image in accordance with the second NR information. As aresult, noise reduction processing information, which defines noisereduction processing according to a specific noise reduction processingmethod, can be used when carrying out noise reduction processingaccording to a different noise reduction processing method.

The present embodiment describes a case where the digital video camera100 and the image playback apparatus 200 each handle amutually-different noise reduction processing method as an example.However, the technique of the present embodiment can also be applied ina case where the digital video camera 100 and the image playbackapparatus 200 execute noise reduction processing using a plurality ofthe same noise reduction processing methods, but in different orders.For example, a case is conceivable in which the digital video camera 100carries out noise reduction processing in the order of epsilon filterprocessing and median filter processing, whereas the image playbackapparatus 200 carries out noise reduction processing in the order ofmedian filter processing and epsilon filter processing. In such a case,if the epsilon filter processing and the median filter processing, andthe order in which they are carried out, are taken as a single noisereduction processing method, the digital video camera 100 and the imageplayback apparatus 200 can be considered to be using different noisereduction processing methods. In this case, the first NR informationincludes the parameters for both the epsilon filter processing and themedian filter processing. Then, on the basis of the first NRinformation, the image playback apparatus 200 may generate the second NRinformation, including the parameters for both the median filterprocessing and the epsilon filter processing, suited to a situationwhere the median filter processing and the epsilon filter processing areexecuted in that order.

Second Embodiment

While the first embodiment described a configuration in which the imageplayback apparatus 200 generates the first NR information and the secondNR information, the second embodiment will describe a configuration inwhich the digital video camera 100 generates the second NR informationfrom the first NR information. In the second embodiment, the basicconfigurations of the digital video camera 100 and the image playbackapparatus 200 are the same as in the first embodiment (see FIGS. 1 and2). The following will primarily describe points that are different fromthe first embodiment.

FIG. 7 is a diagram illustrating, in detail, the configurations of theimage processing circuit 24 and the system control circuit 50, accordingto the second embodiment. A second NR information generating circuit 703generates noise reduction processing information for epsilon filterprocessing, which is spatial filter-type noise reduction processing (thesecond NR information), on the basis of the first NR informationgenerated by the first NR information generating circuit 302. Thespecific processing by which the second NR information generatingcircuit 703 generates the second NR information is the same as theprocess described in the first embodiment with reference to step S506 inFIG. 5. However, the second NR information generating circuit 703 maygenerate name and version information indicating the image playbackapparatus (or playback application or the like) for which the second NRinformation is assumed to be used, information indicating the type ofthe noise reduction processing corresponding to the second NRinformation, and the like, and include that information in the second NRinformation.

A metadata generating circuit 704 generates metadata to be added to theRAW moving image. The metadata includes the first NR information, thesecond NR information, and various types of developing parameters. Anadder 705 adds the metadata generated by the metadata generating circuit704 to the RAW moving image input to the image processing circuit 24.

Note that like the first embodiment, the image to be played back by theimage playback apparatus 200 is not limited to a RAW moving image. Forexample, the image to be played back may be a moving image that hasalready been developed. Furthermore, the image to be played back may bea RAW still image, or a still image that has already been developed. Thetechnique of the present embodiment can be applied to any situation inwhich it is necessary to change the type, strength settings, and so onof noise reduction processing between when an image is shot and when theimage is played back.

Furthermore, although the present embodiment describes a case where thesecond NR information is recorded in the metadata of the RAW movingimage to be recorded as an example, the recording destination of thesecond NR information is not limited thereto, and a configuration may beemployed in which the second NR information is added to an ancillarysignal region of an SDI signal, for example.

Additionally, in the present embodiment, the digital video camera 100 isconfigured to be capable of developing/playing back the RAW movingimage. The digital video camera 100 is also configured to be capable ofusing the image processing circuit 24 to carry out developing processingand noise reduction processing on the RAW moving image stored in therecording medium 90, and output the developed video.

FIG. 9 is a conceptual diagram of a RAW moving image to which themetadata has been added. A RAW moving image 901 is constituted by threeframes' worth of images (frames 1 to 3). Metadata 902 includesparameters for the noise reduction processing to be applied, on aframe-by-frame basis. The RAW moving image 901 and the metadata 902 arethe same as the RAW moving image 401 and the metadata 402 described inthe first embodiment with reference to FIG. 4. Metadata 903 includes thesecond NR information generated by the second NR information generatingcircuit 703. As illustrated in FIG. 9, the metadata 903 (second NRinformation) includes parameters for epsilon filter processing, for eachof the frames. P1 and P2 indicated in FIG. 9 are the same as P1 and P2described in the first embodiment with reference to FIG. 6. In theexample illustrated in FIG. 9, an instruction to strengthen the noisereduction processing is made at the timing of frame 2, and the detailsof the recording parameters are changed. The instruction to strengthenthe noise reduction processing is made by the user through the operationmember 70 or the like.

Next, an NR information recording process executed by the digital videocamera 100 will be described with reference to FIG. 8. When the digitalvideo camera 100 is instructed to record a moving image, the digitalvideo camera 100 executes the processing of this flowchart for eachframe.

In step S801, the first NR information generating circuit 302 obtainssetting information for the noise reduction processing to be applied tothe monitoring video. The obtained setting information is menu settingvalues such as “weak”, “strong”, or the like in a noise reduction menu,selected in accordance with a user instruction.

In step S802, the first NR information generating circuit 302 generatesthe first NR information, which is setting values for a recursive filterto be applied to the monitoring video, in accordance with the menusetting values. The first NR information generating circuit 302 convertsthe menu setting values, from weak→R1, strong→R2, and so on. The firstNR information generating circuit 302 outputs the first NR informationto the second NR information generating circuit 703 and the metadatagenerating circuit 704.

In step S803, the second NR information generating circuit 703 generatesthe second NR information on the basis of the first NR information,through the same process as in step S506 of FIG. 5. The second NRinformation generating circuit 703 outputs the generated second NRinformation to the metadata generating circuit 704. Note that the secondNR information generating circuit 703 may generate name and versioninformation indicating the image playback apparatus (or playbackapplication or the like) for which the second NR information is assumedto be used, information indicating the type of the noise reductionprocessing corresponding to the second NR information, and the like, andinclude that information in the second NR information. Additionally, ifit is assumed that an image playback apparatus having a plurality ofnoise reduction processing functions will play the image back,information making it possible to determine the processing order may beincluded in the second NR information. Through this, the setting valuesfor the noise reduction processing circuits on the playback side can bedetermined taking into account the types, processing orders, settingvalues, and the like of the noise reduction processing circuits, even inan apparatus aside from a specific image playback apparatus.

In step S804, the metadata generating circuit 704 generates metadatacontaining the first NR information and the second NR information, andoutputs the metadata to the adder 705. As a result, the metadata isrecorded in the recording medium 90 along with the RAW moving image.

Through the foregoing processing, noise reduction processing can becarried out in accordance with the second NR information when the imageplayback apparatus 200 plays back a RAW moving image. As a result, anoise reduction effect similar to the effect of the noise reductionprocessing which has been carried out by the digital video camera 100 onthe monitoring video when shooting the RAW moving image is also achievedin the moving image played back by the image playback apparatus 200.

Note that like the first embodiment, it is not absolutely necessary forthe effect of the noise reduction processing executed by the second NRcircuit 205 according to the second NR information to become similar tothe effect of the noise reduction processing executed by the first NRcircuit 301 according to the first NR information. Regardless of theformat, an effect in which the first NR information can be utilized inthe execution of the noise reduction processing by the second NR circuit205 is achieved as long as the second NR information is generated on thebasis of the first NR information.

Additionally, the descriptions of the present embodiment assume that thefirst NR circuit 301 actually carries out the noise reduction processingin accordance with the first NR information. However, it is notabsolutely necessary for the first NR circuit 301 to carry out the noisereduction processing in accordance with the first NR information. Theeffects of the present embodiment can be achieved as long as the digitalvideo camera 100 is configured so that the second NR information, whichdefines noise reduction processing carried out through a specific noisereduction processing method applicable to an external apparatus such asthe image playback apparatus 200, is generated on the basis of the firstNR information.

Additionally, although the foregoing descriptions assume the metadatacontains the first NR information and the second NR information, thefirst NR information need not be contained in the metadata. If themetadata contains the first NR information, the digital video camera 100can carry out noise reduction processing in accordance with the first NRinformation when playing back a RAW moving image.

According to the second embodiment as described thus far, the digitalvideo camera 100 generates the first NR information, which defines noisereduction processing applied to a RAW moving image for generating themonitoring video. Then, on the basis of the first NR information, thedigital video camera 100 generates the second NR information, whichdefines noise reduction processing using a different noise reductionprocessing method from that used by the digital video camera 100, andrecords the second NR information and the RAW moving image inassociation with each other. As a result, noise reduction processinginformation, which defines noise reduction processing according to aspecific noise reduction processing method, can be used when carryingout noise reduction processing according to a different noise reductionprocessing method.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2019-025787, filed Feb. 15, 2019 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: atleast one memory configured to store instructions; and at least oneprocessor and/or at least one circuit in communication with the at leastone memory and configured to execute the instructions to: obtain animage and first NR information defining noise reduction processing usinga first noise reduction processing method that an external apparatus canapply to the image; generate, on the basis of the first NR information,second NR information defining noise reduction processing using a secondnoise reduction processing method that is different from the first noisereduction processing method; and apply the noise reduction processingusing the second noise reduction processing method to the image inaccordance with the second NR information, wherein the second NRinformation is generated such that an effect of the noise reductionprocessing using the second noise reduction processing method accordingto the second NR information becomes similar to an effect of the noisereduction processing using the first noise reduction processing methodaccording to the first NR information.
 2. The image processing apparatusaccording to claim 1, wherein the image is a moving image including aplurality of frames, wherein the first NR information defines the noisereduction processing using the first noise reduction processing methodfor each of the plurality of frames, and wherein the second NRinformation is generated so as to define the noise reduction processingusing the second noise reduction processing method for each of theplurality of frames.
 3. The image processing apparatus according toclaim 2, wherein the first noise reduction processing method includesone of a recursive noise reduction processing method and a spatialfilter-type noise reduction processing method, and wherein the secondnoise reduction processing method includes the other of the recursivenoise reduction processing method and the spatial filter-type noisereduction processing method.
 4. The image processing apparatus accordingto claim 1, wherein the second noise reduction processing method carriesout a same plurality of types of noise reduction processing as the firstnoise reduction processing method, and wherein an order in which theplurality of types of noise reduction processing are carried out isdifferent for the first noise reduction processing method and the secondnoise reduction processing method.
 5. The image processing apparatusaccording to claim 1, wherein the at least one processor furtherexecutes the instructions to obtain metadata that contains the first NRinformation and that is associated with the image.
 6. An imageprocessing method executed by an image processing apparatus, the methodcomprising: obtaining an image and first NR information defining noisereduction processing using a first noise reduction processing methodthat an external apparatus can apply to the image; generating, on thebasis of the first NR information, second NR information defining noisereduction processing using a second noise reduction processing methodwhich is different from the first noise reduction processing method; andapplying the noise reduction processing using the second noise reductionprocessing method to the image in accordance with the second NRinformation, wherein the second NR information is generated such that aneffect of the noise reduction processing using the second noisereduction processing method according to the second NR informationbecomes similar to an effect of the noise reduction processing using thefirst noise reduction processing method according to the first NRinformation.
 7. A non-transitory computer-readable storage medium whichstores a program for causing a computer to execute an image processingmethod comprising: obtaining an image, and first NR information definingnoise reduction processing using a first noise reduction processingmethod which an external apparatus can apply to the image; generating,on the basis of the first NR information, second NR information definingnoise reduction processing using a second noise reduction processingmethod which is different from the first noise reduction processingmethod; and applying the noise reduction processing using the secondnoise reduction processing method to the image in accordance with thesecond NR information, wherein the second NR information is generatedsuch that an effect of the noise reduction processing using the secondnoise reduction processing method according to the second NR informationbecomes similar to an effect of the noise reduction processing using thefirst noise reduction processing method according to the first NRinformation.